{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "This example demonstrates how to convert a network from [Caffe's Model Zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo) for use with Lasagne. We will be using the Lenet trained for MNIST.\n",
    "\n",
    "We will create a set of Lasagne layers corresponding to the Caffe model specification (prototxt), then copy the parameters from the caffemodel file into our model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Converting from Caffe to Lasagne\n",
    "### Download the required files\n",
    "\n",
    "First we download `cifar10_nin.caffemodel` and `model.prototxt`. The supplied `train_val.prototxt` was modified to replace the data layers with an input specification, and remove the unneeded loss/accuracy layers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Caffe\n",
    "\n",
    "To load the saved parameters, we'll need to have Caffe's Python bindings installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "caffe_root = '/home/xilinx/caffe/'  # this file should be run from {caffe_root}/examples (otherwise change this line)\n",
    "sys.path.insert(0, caffe_root + 'python')\n",
    "import caffe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the pretrained Caffe network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net_caffe = caffe.Net('lenet.prototxt', 'lenet_iter_10000.caffemodel', caffe.TEST)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Lasagne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import lasagne\n",
    "from lasagne.layers import InputLayer, DropoutLayer, DenseLayer, NonlinearityLayer\n",
    "#from lasagne.layers.dnn import Conv2DDNNLayer as ConvLayer\n",
    "from lasagne.layers import Conv2DLayer as ConvLayer\n",
    "from lasagne.layers import Pool2DLayer as PoolLayer\n",
    "from lasagne.nonlinearities import softmax, rectify, linear\n",
    "import conv_fpga\n",
    "from conv_fpga import FPGA_LENET\n",
    "from conv_fpga import FPGAQuickTest\n",
    "#from conv_fpga import Conv2DLayer as ConvLayer\n",
    "from conv_fpga import FPGAWeightLoader as FPGALoadW\n",
    "from lasagne.utils import floatX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a Lasagne network\n",
    "Layer names match those in `model.prototxt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net = {}\n",
    "net['input'] = InputLayer((None, 1, 28, 28))\n",
    "net['conv1'] = ConvLayer(net['input'], num_filters=20, filter_size=5, nonlinearity=linear)\n",
    "net['pool1'] = PoolLayer(net['conv1'], pool_size=2, stride=2, mode='max', ignore_border=False)\n",
    "net['conv2'] = ConvLayer(net['pool1'], num_filters=50, filter_size=5, nonlinearity=linear)\n",
    "net['pool2'] = PoolLayer(net['conv2'], pool_size=2, stride=2, mode='max', ignore_border=False)\n",
    "net['ip1'] = DenseLayer(net['pool2'], num_units=500, nonlinearity = rectify)\n",
    "net['ip2'] = DenseLayer(net['ip1'], num_units=10, nonlinearity = None)\n",
    "net['prob'] = NonlinearityLayer(net['ip2'], softmax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Copy the parameters from Caffe to Lasagne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "layers_caffe = dict(zip(list(net_caffe._layer_names), net_caffe.layers))\n",
    "\n",
    "for name, layer in net.items():\n",
    "    try:\n",
    "        if name=='ip1'or name=='ip2':\n",
    "            layer.W.set_value(np.transpose(layers_caffe[name].blobs[0].data))\n",
    "            layer.b.set_value(layers_caffe[name].blobs[1].data)\n",
    "        else:\n",
    "            layer.W.set_value(layers_caffe[name].blobs[0].data[:,:,::-1,::-1])\n",
    "            layer.b.set_value(layers_caffe[name].blobs[1].data)\n",
    "            \n",
    "    except AttributeError:\n",
    "        continue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Copy the parameters from CPU to FPGA OnChip Memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight shape (20, 1, 5, 5)\n",
      "Loading Started for Layer  1\n",
      "Elapsed Test Time:  0.002136251999999672\n",
      "Loading Finished for Layer  1\n",
      "weight shape (50, 20, 5, 5)\n",
      "Loading Started for Layer  2\n",
      "Elapsed Test Time:  0.0007430980000009413\n",
      "Loading Finished for Layer  2\n",
      "weight shape (500, 50, 4, 4)\n",
      "Loading Started for Layer  3\n",
      "Elapsed Test Time:  0.009178858000002066\n",
      "Loading Finished for Layer  3\n",
      "weight shape (10, 500, 1, 1)\n",
      "Loading Started for Layer  4\n",
      "Elapsed Test Time:  0.00025717900000188365\n",
      "Loading Finished for Layer  4\n"
     ]
    }
   ],
   "source": [
    "#FPGALoadW(weight, status, IFDim, OFDim, PadDim)\n",
    "weight = net['conv1'].W.get_value()\n",
    "FPGALoadW(weight, 1, 28, 24, 0)\n",
    "weight = net['conv2'].W.get_value()\n",
    "FPGALoadW(weight, 2, 12, 8, 0)\n",
    "weight = net['ip1'].W.get_value()\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(500, 50, 4, 4)\n",
    "FPGALoadW(weight, 3, 4, 1, 0, flip_filters=False)\n",
    "weight = net['ip2'].W.get_value()\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(10, 500, 1, 1)\n",
    "FPGALoadW(weight, 4, 1, 1, 0, flip_filters=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Trying it out\n",
    "Let's see if that worked. \n",
    "\n",
    "### Import numpy and set up plotting\n",
    "### Import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import gzip\n",
    "import _pickle as cPickle\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download some test data\n",
    "Load test mnist handwritting data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x2a105d50>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATYAAAEzCAYAAAC7cS8aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADkdJREFUeJzt3WGMVfWZx/HfT7GJLYlBImBEYLXZrtnEjBiMGzSB0G2J\nvsAocY19od0EfYHWhITU+oa31RdkdRNNVtGgwXRZjUKMsVhJNFZpkYqCIlQMAsUZycI2YkKE5dkX\n97hM6Z3Of+45d+7cZ76fhHjnzDNz/9eDX8+9c88cR4QAIJPzer0AAGgaYQOQDmEDkA5hA5AOYQOQ\nDmEDkM6UOl9se6mkf1MrkOsi4uE2M7yfBEBXRITbbXen72OzfZ6kfZKWSDoiabukOyLik3PmCBuA\nrhgpbHWeil4n6Y8R8XlEnJL0K0nLanw/AGhEnbBdJunQsI8PV9sAoKf44QGAdOqE7U+S5gz7eHa1\nDQB6qk7Ytkv6vu25tr8j6Q5Jm5tZFgB0ruO3e0TE/9q+T9IWnX27x57GVgYAHer47R7Fd8DbPQB0\nSTfe7gEAExJhA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5h\nA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmED\nkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5DOlDpf\nbPuApD9LOiPpVERc18SiAKCOWmFTK2iLIuJ4E4sBgCbUfSrqBr4HADSqbpRC0uu2t9te0cSCAKCu\nuk9FF0bEF7YvUStweyLi7SYWBgCdqnXEFhFfVP88KuklSfzwAEDPdRw229+1PbW6/T1JP5K0u6mF\nAUCn6jwVnSnpJdtRfZ8NEbGlmWUBQOccEd29g1b4AKBxEeF223mrBoB0CBuAdAgbgHQIG4B0CBuA\ndAgbgHQIG4B0CBuAdAgbgHQIG4B06v7aIpxj+fLlRXMrVpT9+rojR44UzZ08ebJobsOGDUVzg4OD\nRXOffvpp0RwwnjhiA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA5hA5AOYQOQDmEDkA4Xc2nYZ599VjQ3\nb9687i6kpq+++qpo7qOPPurySnI7fPhw0dwjjzxSNPfee+/VWU7f4WIuACYNwgYgHcIGIB3CBiAd\nwgYgHcIGIB3CBiAdwgYgHcIGIB2uedCw0msZXH311UVze/bsKZq76qqriubmz59fNLdo0aKiueuv\nv75o7tChQ0Vzl19+edFc006fPl00d/To0aK5Sy+9tM5y/srBgweL5ibbmQcj4YgNQDqEDUA6hA1A\nOoQNQDqEDUA6hA1AOoQNQDqEDUA6hA1AOlzzAG1NmzataG5gYKBobseOHUVzCxYsKJpr2smTJ4vm\n9u3bVzRXesbIxRdfXDS3cuXKorknnniiaC6Ljq95YHud7SHbHw7bNs32Ftt7bf/a9kVNLhYA6ih5\nKvqMpB+fs+1BSb+JiB9I2irpF00vDAA6NWrYIuJtScfP2bxM0vrq9npJtzS8LgDoWKc/PJgREUOS\nFBGDkmY0tyQAqKepn4ryAwIAE0anYRuyPVOSbM+S9GVzSwKAekrD5urPtzZLuru6fZekTQ2uCQBq\nKXm7x/OS3pH097YP2v6ppF9K+mfbeyUtqT4GgAlh1F8NHhF3jvCpHza8FgBoBGceAGNw2223Fc1t\n3LixaG737t1Fc4sXLy6aO3bsWNFcFh2feQAA/YawAUiHsAFIh7ABSIewAUiHsAFIh7ABSIewAUiH\nsAFIhzMPAEkzZpT9SsFdu3Y1+v2WL19eNPfiiy8WzU02nHkAYNIgbADSIWwA0iFsANIhbADSIWwA\n0iFsANIhbADSIWwA0hn1Yi7AZLBy5cqiuUsuuaRo7vjx40Vze/fuLZrD2HDEBiAdwgYgHcIGIB3C\nBiAdwgYgHcIGIB3CBiAdwgYgHcIGIB2ueYDUFi5cWDS3devWorkLLrigaG7RokVFc2+99VbRHNrj\nmgcAJg3CBiAdwgYgHcIGIB3CBiAdwgYgHcIGIB3CBiAdwgYgHa55gNRuuummornSMwreeOONorl3\n3323aA7dMeoRm+11todsfzhs2xrbh23/ofqztLvLBIByJU9Fn5H04zbb10bE/OrPaw2vCwA6NmrY\nIuJtSe2uJdb25FMA6LU6Pzy4z/ZO20/ZvqixFQFATZ2G7XFJV0TEgKRBSWubWxIA1NNR2CLiaJz9\nRW5PSlrQ3JIAoJ7SsFnDXlOzPWvY526VtLvJRQFAHaO+j83285IWSZpu+6CkNZIW2x6QdEbSAUn3\ndnGNADAmo4YtIu5ss/mZLqwFABrBmQfoSxdeeGHR3NKlZe8d/+abb4rm1qxZUzR36tSpojl0B+eK\nAkiHsAFIh7ABSIewAUiHsAFIh7ABSIewAUiHsAFIh7ABSIczD9CXVq9eXTR3zTXXFM299lrZL4F+\n5513iubQWxyxAUiHsAFIh7ABSIewAUiHsAFIh7ABSIewAUiHsAFIh7ABSMdnLw/apTuwu3sHSOXm\nm28umnv55ZeL5r7++uuiudJrI2zbtq1oDuMjItxuO0dsANIhbADSIWwA0iFsANIhbADSIWwA0iFs\nANIhbADSIWwA0uGaBxgX06dPL5p77LHHiubOP//8orlXX321aI4zCnLhiA1AOoQNQDqEDUA6hA1A\nOoQNQDqEDUA6hA1AOoQNQDqEDUA6o17zwPZsSc9KminpjKQnI+Ix29Mk/aekuZIOSLo9Iv7c5uu5\n5kFipWcAlL6z/9prry2a279/f9Fc6bUMSr8fJpY61zw4LWlVRPyjpH+StNL2P0h6UNJvIuIHkrZK\n+kVTiwWAOkYNW0QMRsTO6vYJSXskzZa0TNL6amy9pFu6tUgAGIsxvcZme56kAUnbJM2MiCGpFT9J\nM5peHAB0ojhstqdKekHSA9WR27mvnfFaGoAJoShstqeoFbXnImJTtXnI9szq87MkfdmdJQLA2JQe\nsT0t6eOIeHTYts2S7q5u3yVp07lfBAC9MOovmrS9UNJPJO2y/b5aTzkfkvSwpI22/1XS55Ju7+ZC\nAaDUqGGLiN9KGunNSj9sdjkAUB9nHgBIh2seoJYrr7yyaK70jIJSq1atKprjjILJiSM2AOkQNgDp\nEDYA6RA2AOkQNgDpEDYA6RA2AOkQNgDpEDYA6XDmAdqaO3du0dyWLVsavd/Vq1cXzb3yyiuN3i9y\n4YgNQDqEDUA6hA1AOoQNQDqEDUA6hA1AOoQNQDqEDUA6hA1AOpx5gLbuueeeork5c+Y0er9vvvlm\n0VxENHq/yIUjNgDpEDYA6RA2AOkQNgDpEDYA6RA2AOkQNgDpEDYA6RA2AOlw5sEkc8MNNxTN3X//\n/V1eCdA9HLEBSIewAUiHsAFIh7ABSIewAUiHsAFIh7ABSIewAUiHsAFIZ9QzD2zPlvSspJmSzkj6\nj4j4d9trJK2Q9GU1+lBEvNa1laIRN954Y9Hc1KlTG73f/fv3F82dOHGi0fvF5FRyStVpSasiYqft\nqZJ22H69+tzaiFjbveUBwNiNGraIGJQ0WN0+YXuPpMuqT7uLawOAjozpNTbb8yQNSPpdtek+2ztt\nP2X7oobXBgAdKQ5b9TT0BUkPRMQJSY9LuiIiBtQ6ouMpKYAJoShstqeoFbXnImKTJEXE0Th71don\nJS3ozhIBYGxKj9ielvRxRDz67Qbbs4Z9/lZJu5tcGAB0quTtHgsl/UTSLtvvSwpJD0m60/aAWm8B\nOSDp3i6uEwCKlfxU9LeSzm/zKd6zBmBC4swDAOlwzQPU8sEHHxTNLVmypGju2LFjdZYDSOKIDUBC\nhA1AOoQNQDqEDUA6hA1AOoQNQDqEDUA6hA1AOoQNQDo++5uHunQHdnfvAMCkFRFtf4s3R2wA0iFs\nANIhbADSIWwA0iFsANIhbADSIWwA0iFsANIhbADSIWwA0un6KVUAMN44YgOQDmEDkM64hs32Utuf\n2N5n++fjed9Nsn3A9ge237f9+16vp5TtdbaHbH84bNs021ts77X9a9sX9XKNJUZ4HGtsH7b9h+rP\n0l6usYTt2ba32v7I9i7bP6u2980+afMY7q+293R/jNtrbLbPk7RP0hJJRyRtl3RHRHwyLgtokO3P\nJF0bEcd7vZaxsH2DpBOSno2Iq6ttD0v674h4pPqfzbSIeLCX6xzNCI9jjaSvImJtTxc3BrZnSZoV\nETttT5W0Q9IyST9Vn+yTv/EY/kU93B/jecR2naQ/RsTnEXFK0q/U+hfQj6w+fBofEW9LOjfGyySt\nr26vl3TLuC6qAyM8Dqm1X/pGRAxGxM7q9glJeyTNVh/tkxEew2XVp3u2P8bzP87LJB0a9vFhnf0X\n0G9C0uu2t9te0evF1DQjIoak1l9SSTN6vJ467rO90/ZTE/npWzu250kakLRN0sx+3CfDHsPvqk09\n2x99d9QxQSyMiPmSbpK0snpqlEW/vv/ncUlXRMSApEFJ/fSUdKqkFyQ9UB31nLsPJvw+afMYero/\nxjNsf5I0Z9jHs6ttfScivqj+eVTSS2o9ze5XQ7ZnSv//esmXPV5PRyLiaJx9wfhJSQt6uZ5Stqeo\nFYTnImJTtbmv9km7x9Dr/TGeYdsu6fu259r+jqQ7JG0ex/tvhO3vVv93ku3vSfqRpN29XdWYWH/5\n2sdmSXdXt++StOncL5ig/uJxVAH41q3qn33ytKSPI+LRYdv6bZ/81WPo9f4Y1zMPqh/5PqpWUNdF\nxC/H7c4bYvvv1DpKC0lTJG3ol8dh+3lJiyRNlzQkaY2klyX9l6TLJX0u6faI+J9erbHECI9jsVqv\n75yRdEDSvd++TjVR2V4o6S1Ju9T6+xSSHpL0e0kb1Qf75G88hjvVw/3BKVUA0uGHBwDSIWwA0iFs\nANIhbADSIWwA0iFsANIhbADSIWwA0vk/hwd/8KUsrxAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a241bb0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = np.load('test_cpr.npz')\n",
    "test_data = data['data'].reshape(10000, 1, 28, 28)\n",
    "test_label = data['label']\n",
    "##\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.imshow(test_data[0][0], interpolation='nearest', cmap=plt.get_cmap('gray'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FPGA Deployment (Lasagne Layer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "FPGA_net = {}\n",
    "FPGA_net['input'] = InputLayer((None, 1, 28, 28))\n",
    "FPGA_net['lenet'] = FPGA_LENET(FPGA_net['input'])\n",
    "FPGA_net['prob'] = NonlinearityLayer(FPGA_net['lenet'], softmax)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  0.9378025709999989\n",
      "CPU times: user 3.34 s, sys: 1.49 s, total: 4.83 s\n",
      "Wall time: 13.1 s\n"
     ]
    }
   ],
   "source": [
    "batch_size = 500\n",
    "\n",
    "%time prob = lasagne.layers.get_output(FPGA_net['prob'], floatX(test_data[0:batch_size]), deterministic=True).eval()\n",
    "FPGA_predicted = np.argmax(prob, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.984\n"
     ]
    }
   ],
   "source": [
    "FPGA_accuracy = np.mean(FPGA_predicted == test_label[0][0:batch_size])\n",
    "#print(FPGA_predicted)\n",
    "#print(test_label[0][0:600])\n",
    "print(FPGA_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FPGA Deployment (QuickTest Function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  1.125952114999997\n",
      "CPU times: user 1.19 s, sys: 50 ms, total: 1.24 s\n",
      "Wall time: 1.24 s\n"
     ]
    }
   ],
   "source": [
    "batch_size = 600\n",
    "\n",
    "OFMDim = 1\n",
    "OFMCH = 10\n",
    "%time FPGA_output = FPGAQuickTest(test_data, batch_size, OFMDim, OFMCH)\n",
    "FPGA_predicted = np.argmax(FPGA_output.reshape(batch_size, -1), 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.983333333333\n"
     ]
    }
   ],
   "source": [
    "FPGA_accuracy = np.mean(FPGA_predicted == test_label[0][0:batch_size])\n",
    "print(FPGA_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph some images and predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5sAAABuCAYAAACp3AcnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VFX6P/DPEBAikaahFylGAgkkIEU2soo0qUZhDSAq\nTXYVRWzo6rrYsKBo6GUtgEhRUZDqokQhwvIDgy4BzA8QQpCqSAtSzHz/YM/JM8yEJDN3zs3c+bz3\nxYvHM5m5z55c7syd+9znuNxuN4iIiIiIiIisVMruBIiIiIiIiMh5gnKyGRkZedDlcrn9+RMZGXkw\nGDk5lb9zzXkuPs61OZxrM3isNodzbQ6PH+Zwrs3hXJvBY7X1XMEoo3W5XG5/X9flcsHtdrssTsmx\n/J1rznPxca7N4VybwWO1OZxrc3j8MIdzbQ7n2gweq63HMloiIiIiIiKyHE82iYiIiIiIyHI82SQi\nIiIiIiLLGTvZzMrKQmJiIlq0aIHExERUrFgREyZMMLX5sJKTk4MOHTqgadOmiI+P5zwH0ZAhQ1Ct\nWjU0a9bM7lQcb+XKlWjcuDFiYmLw2muv2Z2O4+Xl5aFFixbo1auX3ak4Fo8fZnGfNiM1NRXx8fH8\n/GHAtddei+bNmyMxMRGtW7e2Ox1H437tP2MnmzExMcjIyMB3332HzZs3o3z58khOTja1+bBSunRp\njB8/HpmZmVi/fj0mT56MHTt22J2WIw0aNAirVq2yOw3Hy8vLw4gRI7Bq1SpkZmZi3rx53KeDLDU1\nFU2aNLE7DUfj8cMs7tPBl5mZiXfeeQebNm3Cli1bsHTpUuzevdvutByrVKlSSEtLQ0ZGBjZu3Gh3\nOo7F/TowtpTRrl69Gg0bNkSdOnXs2LzjVa9eHQkJCQCAqKgoxMbGYv/+/TZn5UxJSUmoXLmy3Wk4\n3saNG3HdddehXr16KFOmDFJSUrB48WK703KsnJwcLF++HEOHDrU7FUfj8cMc7tNmbN++HW3atEHZ\nsmURERGB9u3bY9GiRXan5Vhutxt5eXl2p+F43K8DY8vJ5oIFC9CvXz87Nh129uzZgy1btqBNmzZ2\np0Lkt/3793t8OVW7dm1+gRJEo0aNwrhx4+BysYM7OQP3aTPi4uKwdu1aHDt2DLm5uVi+fDn27dtn\nd1qO5XK50KlTJ7Rq1QozZ860Ox3H4n4dmNKmN3j+/HksWbIEr776qulNh51Tp06hT58+SE1NRVRU\nlN3pEFEIWLZsGapVq4aEhASkpaUhGGsxE5nEfdqcxo0bY/To0ejUqROioqKQmJiIiIgIu9NyrPT0\ndNSoUQNHjhxBp06dEBsbi6SkJLvTchzu14ExfmVzxYoVaNmyJaKjo01vOqxcuHABffr0wcCBA9G7\nd2+70yEKSK1atZCdna3/OycnB7Vq1bIxI+dKT0/HkiVL0KBBA/Tr1w9r1qzBPffcY3daRH7jPm3W\noEGDsGnTJqSlpaFSpUqIiYmxOyXHqlGjBgAgOjoaycnJvG8ziLhfB8Dtdlv+5+LL+paSkuJ+//33\nC3wcQNj/sWKuBw4c6B41ahTn2aJ5vtxc//TTT+64uDjOdRDn+sKFC+6GDRu69+zZ4z579qy7efPm\n7m3btnGuA5zrS+f5Umlpae6ePXtyvw7iXBd2/OBcW7tfc5+2bp4LmuvDhw+73W63e+/eve7Y2Fj3\n8ePHOddBmOvTp0+7T5486Xa73e5Tp06527Vr5161ahXnOsC5vnSeuV8HPtdGr2zm5uZi9erVuOOO\nO0xuNuykp6dj7ty5+Oqrr/RyMytXrrQ7LUfq378/2rVrh6ysLNStWxfvvfee3Sk5UkREBCZNmoTO\nnTujadOmSElJQWxsrN1pEQWExw9yojvvvBNxcXHo3bs3pkyZggoVKtidkiMdOnQISUlJSExMRNu2\nbdGzZ0907tzZ7rQci/u1/1z/O4u39kVdLre/r8ub9wG3213kSfB3rjnPxZtngHMdCM61OSaOH/97\nrl/PcxLOtTl8XzSDx2pzONfm8Fhtjq+5tqUbLRERERERETkbTzaJiIiIiIjIcjzZJCIiIiIiIsvx\nZJOIiIiIiIgsVzoYL1quXLlDLpermj/PLVu2LM6ePWt1So7l71wHOs+PP/64jiMjI3XcrFkzAECf\nPn18Pm/q1KkAgPXr1+uxOXPm+J2HSXbNdTjiXJvBY7U5nGtzePwwh3NtDufaDB6rrReUk80zZ85U\nV7HL5SpWSyf+korH37nmPBcf59ocNdc8fgQXj9XmcK7N4bHaHM61OXxfNIPHauuxjJaIiIiIiIgs\nF5R1Nj02UMxvBaj4ay8pwZ7rBQsW6LigMtmi2rVrl447duyo4+zs7IBetzj8nWcgtPbrmJgYHe/Y\nsUPHI0eO1PHEiRODmkNJ3aeLonz58joeN24cAGD48OF6bPPmzTru27evjvfu3WsgO2+hPNehhnNt\nDufajHB5XywJwnGfrly5MgCgbt26hf6seg8dNWqUHtu6dauOs7KydPz9999f9rXCca7twnU2iYiI\niIiIyAiebBIREREREZHlgtIgiJxFlc8WpXRWlWmuWrVKjzVo0EDHPXv2BAA0bNhQjw0YMEDHr7zy\nSmDJkpfExEQd5+Xl6TgnJ8eOdEJOjRo1dDxs2DAAnvPYsmVLHffo0UPHkydPNpBdaGrRooWOFy1a\npONrr73Wsm107txZx9u3b9fxvn37LNtGuFDHbQBYsmQJAGDEiBF6bNq0aTr+448/zCVWAlStWlXH\nCxcu1PG3336r4xkzZgAA9uzZE5QcKlasqOP27dsDAFauXKnHzp8/H5TtEhWke/fuOu7Vq5eOb775\nZgBAo0aNCn0NVSZbr149PVa2bFmfPxsREeFPmmQIr2wSERERERGR5XiySURERERERJZjGS35dMMN\nN+g4OTnZ6/HMzEwdyxKJo0ePAgBOnTqlx6644godb9iwAQDQvHlzPXb11VdbkDEVJCEhQcenT5/W\n8aeffmpHOiEhOjpax7NmzbIxE2fq0qWLjgsqiwqULP0cPHiwjlNSUoKyPaeRx+UpU6Z4PT5p0iQd\nv/vuuzo+c+ZMcBMrIVRXTfleKMtZDx06pONglM/KbcmO2OrYJcv7d+7cafn2S5IKFSroWN2KExcX\np8dkx3uWFFtD3gr14IMPAsi/zQQAIiMjdexy+dfgWHbSp9DGK5tERERERERkuRJ/ZVM2pZHfmvz8\n8886/v333wEAc+fO1WMHDx7UsdO/1QsG2RRFfSslv8GVVyYOHDhw2dd67LHHdNykSROvx5ctW+Z3\nnlQw9c2ubOQxZ84cu9Ip8R5++GEd33777Tpu3bp1kV9DNecAgFKlLn6XJ9f/+uabbwJJMeSVLn3x\nLadbt25B35a82vPoo4/qWK2bKq/ykze5L9euXdvr8Xnz5ulYvQc73TXXXKNj1TivSpUqekxeAX7o\noYeCmsuzzz6r4/r16+tYrQHs9M89srHgyy+/rOM6dep4/ay88vnLL78EN7EwIY8Jcs3uQMm1wOVn\nTrpINVaSxyJZfagaMAH5jQxlA7f09HQdmzxG8MomERERERERWY4nm0RERERERGS5El9G+/rrr+u4\nsDXYVPkIAJw8eVLHwboUL9cpVHlu2rQpKNsy7fPPP9exumwv5/TXX38t8mvJhhxlypSxIDsqisaN\nGwPILxsE8ku/yNtbb72lY7mOZnHccccdXvHevXv12F133aVjWeYZLm655RYAwI033qjH5DHeSqqB\nC+BZvn/llVcCYBmtL7JZ0zPPPHPZn5Ul+W63O2g5lSRyfVhZrqa88MILQc+hadOmADxvT5HN3px8\njJelm2+//baOZTMrX/vixIkTdaxuKynOZ5hwIUszVWmsLLuUa7eePXtWx8ePHwfgeUyVnzu++OIL\nHW/duhUA8J///EePZWRk6Fg2GAvnY7RscCVvhVKfK+TvqjBt2rTR8YULF3T8448/6njdunUAPEui\nz507V4yML49XNomIiIiIiMhyPNkkIiIiIiIiy5X4MlrZgbZZs2Y63r59u45jY2MBFFzi0rZtWx3v\n27cPgO+OZZdSl5uPHDmix2SXVik7OxuAc8poJVkGWFRPPPGEjn2tlSRLKGRM1nnyyScBeP7+nLh/\nBmr58uUA8rvHFpfsbijXl61Xrx4Az06RGzdu1HFERIRf2ws1shxIdTDdtWuXHhs7dmxQttu7d++g\nvK6TxcfH61iu0yip98UVK1YYycluVatW1fGdd97p9fiQIUN0LD8rWEmVzgLA6tWrvR6XZbTydhen\nefzxx3UsuwAXRt6+0LVrVwCeHWxlma2VpYOhoKByV7UWuq911oH8NdOB/M/ecj3ZunXr6ljecubv\nLSpOps5t1HqlgOc+K7spK/v379fx2rVrdfzTTz/pWH0GlLfsyO768t+Q6hAvu+fLLraB4pVNIiIi\nIiIishxPNomIiIiIiMhyJb6M9ssvv/QZS7JDliI7ESYkJOhYXU5u1apVodtWC1VnZWXpMVm+Ky9B\ny7KwcNajRw8Anl35rrjiCh0fPnwYAPD000/rsdzcXEPZOZ/s2HzDDTcA8Nx/w7m7m/TnP/9Zx9df\nfz0Az/Kewkp9ZHmJLD1SXfkAoEOHDgAK7ur5t7/9TcdTp04tStohSS4+r0q2VCkb4Fl6HCh5TJa/\nY5ZuFY2vMtFLyf09HLz55ps6vvvuu3WsPkt89NFHQc/hpptu0nG1atUAAO+//74e++CDD4Keg53U\nLQmDBg3y+fgPP/yg40OHDgEAOnbs6PNnK1asCMCzJHfu3Lk6PnjwYGDJhgD5mezDDz/UsSqdBfJv\nb/BVtn0pWT6rqFvLyLfp06frWJUqF9RhVp77/Pe//wUA/P3vf9dj6lzlUu3atQPg+Vnj3Xff1bE8\nN1L/biZPnqzHPvnkEx0HeosAr2wSERERERGR5Ur8lU1/HTt2TMdr1qzxerygq6S+yG975RVT9Q0D\n4Oy1rYpDXU2T35xJap6+/vprYzmFE3k1RwlW04pQI6/6zp8/X8eFrVclGyypb/qef/55PVbQlXn1\nvPvvv1+PRUdH61iuL1muXDkAwKRJk/TY+fPnL5tXSdanTx8dq8YDALBz504AwWtUJa8iy6uZaWlp\nOv7tt9+Csm0naN++vc9x2TSlsPU3nUau2yj3qZ9//hmAtQ1lIiMjdSyvXDzwwANe+QwePNiy7ZZ0\n6grMVVddpcdkUxT5vqeOpf369dNjci4bNmwIAKhevboeW7x4sY5vu+02HTttLc6oqCgAnpVlqhoN\nAI4eParjN954AwArzwKl9kcgv2EPAAwdOlTHLpcLgOdnNVntNG7cOB0XpzpNrT8rmxGOGTNGx7Iq\nVFUPBAuvbBIREREREZHleLJJRERERERElnNsGa0V1PpaU6ZM0WNyLT7ZBMdp5RbF8dlnn+m4c+fO\nXo/Pnj1bx7JZCFlPrpOnyHLNcFa6dP7hrrDSWVnmnZKSomNZZlQYVUb7yiuv6LHx48fr+Morr9Sx\n+h0tWbJEj4Vy07G+ffvqWP7/lMdSK6kS6QEDBuixP/74Q8cvvfSSjkO5PDlYVCMJ9felZOnWli1b\njORU0nXv3h2AZ8MkWaJdnKZfqgy0oPXBpY8//rg4aTpC2bJlAXiWNL/11ls+f1Y1S3nvvff0mDwe\nNWjQwOs5slTUyets3n777QCAp556So/JRj6yEZVsdkf+k/+m5frzqnQWyF8zU96yJ9fkLowsk61T\np46O1WdvtZY44HkroKTymTNnjh6z8pYTXtkkIiIiIiIiy/Fkk4iIiIiIiCzHMtrLePDBBwF4dpCU\nXW5//PFH4zmVFDVq1NCxLL1S5S6y3FCWsFm5ph5dJMut5DpkGRkZAIB///vfxnMKRbJDquz0WJzS\nWV9kaaws8yzKWr+hRK1fBxRcAhis9URVx19ZHi3XRPbVkZzyFbYvOnkd2MKkpqbq+JZbbtFxzZo1\nAXh28JWlcb169SryNtTzZJmotHv3bh3LzqrhQnaWVVQZM+B5K48vqkt+QTZs2KBjJ39G8VUmrz4n\nAEBOTo7JdMKCLHGVt3ZIFy5cAAC0adNGj8mO7o0bN/Z6zpkzZ3QcGxvrM1afXdTavJej1tkM1i0n\nvLJJREREREREluPJJhEREREREVmOZbSX+NOf/qRj2bFLUd28AGDr1q1GciqJ1OL2QP7CsdIHH3yg\n41DuqhkKOnbsqOMqVaroWC3Yq7rzUT7ZVVqRJSxWkqV1cru+cpALLg8cODAo+QSLKqEHgFq1aul4\n3rx5Qd+2WqhdCufjc3H5KjP0t7Oq02zevFnHzZo103FCQgIAoGvXrnpMdpuUC7TPmjXrsttQHSC/\n//57n49/++23Og7H91N1DJGlybL0W5YZqo7sycnJekx24FT7tRwbNmyYjmU3zm3btgWce0kiSzMV\nuf/+85//1PHixYsBsPt0oL766isdy9s55Oe2unXrAgAmTJigxwoqqVeluLI8tyC+ymfz8vJ0/Omn\nn+r44YcfBgAcOHCg0Nf1B69sEhERERERkeVcBZ09W7YBlyu4G7DYyy+/rOOnn34aAPDll1/qsW7d\nuuk4WOu1ud1uV+E/5S3Ycy2/VVy4cKGOy5Qpo+O0tDQAQO/evfVYSb3h3t95BkrWfv3RRx/pWK7T\npGL57ZVdSsI+/cYbb+h45MiRXo/L/dhKDz30kI7lOpvyyqb6tlF+Q+/vFQy75joyMlLHa9eu1bGc\nV9VgxYp1idU6yIDvb2PVN7UAMHny5IC350tJ2K/9lZSUpGO1rqzcJ9U6sUD+OqZ2CuW5Loxa+3Hn\nzp16TF5R6tKli47lFdNgKInvi6piR86PbEgmq0d8faZdvXq1jlXjx6VLl+qx6667TsczZ87U8V//\n+tdA0i6U6X1azY28ulUQ9TPTpk3TY7KRkroaB+T/XjIzM32+VtOmTXW8fv16AOabEZW040elSpV0\nrKooZWXlL7/8omO5FqqqIGrevLkea926dZG3K3+fstmYlWtq+pprXtkkIiIiIiIiy/Fkk4iIiIiI\niCzHBkHwLP+SN0ufO3cOgOdN08EqnS3JVAMgecm9oJJDVfpTUktnnaR69eoAgJtuukmPybVfS0L5\nbEnSs2fPoG9DrsnbpEkTAEVbF0+VxoXy8UWu+yVLgGVp97JlywB4lhMXJi4uTseq3BDwLO30VTpX\nlFKxcCYbu/lqVsX1ec157rnnAHjux6NHj9ZxsEtnSzpVdv+Xv/xFj3388cc6liW1ysSJE3Us51I1\nzFu0aJEek80gZcmyajzmlKZM6laSRx99tNCfVceEBx54QI/J2F9qX1a3XAFASkpKwK8bamTZqq9m\npIWZPXu2jgsqoz158iQAz9/3+++/r+OC1v0MBl7ZJCIiIiIiIsvxZJOIiIiIiIgsxzJaeK6NlZiY\nqGO1TqFc4yocPfbYYwA817WSPvvsMx3LkmMKrvvuuw+AZ1fOFStW2JQNAcAzzzyjY9X1sCB79uzR\n8b333gvAs+tcKJPHAdkpsnv37gCKt/bm0aNHdSzLDK+55prLPk+WC5E3X2vuydKu6dOnm0wn7PTt\n21fH99xzD4D8sjfAsxslXSS7ysr9t3///jpW+7AqTQZ8rzX94osv6jg2NlbHsuu+eg11fA51qlxz\nwYIFeuzDDz/UcenS+acEderUAeC7xD4Q6lYT+ft79tlndfzSSy9Zuj2nefLJJwEUrfRYdVM2sdZ1\nYXhlk4iIiIiIiCzHk00iIiIiIiKyXNiW0apyLgD4xz/+oeMTJ07o+IUXXjCaU0lVWOeyESNG6Jhd\naM2pV6+e19ixY8dsyCS8LV++XMfXX399kZ+3bds2Ha9bt87SnOy2Y8cOHcsOkgkJCQCARo0aFfm1\nZNdJadasWToeMGCA1+OyOy5dVLt2bR3L0kNFLrS+adMmIzmFq9tuu81rbOnSpTr+7rvvTKYTcmRJ\nrYyLSh4fZFmpLKO95ZZbAABVqlTRY6ozbihS3Uflv+2YmBifP3vrrbcC8Fx5YMyYMTou6LaqopK3\nV7Rs2TKg13K6oUOH6liVHMuSZykzM1PHsuOy3Xhlk4iIiIiIiCwXdlc21dpiEyZM0GMRERE6llcp\nNmzYYC6xECa/9SvqOoHHjx/3+Rz5LZqvtbMqVaqk48KuuMo1hNQ6W7m5uUXKLxT06NHDa+zzzz+3\nIZPQIL9J9dX0wNeVBgCYMWOGjmvWrOn1uHyt4qztaGLdz5JGrcOr/g7E7t27L/u4XJ9z69atAW/P\nCdq1a6djX/8GZLM3Ci55vDl9+jQA4M0337QrnbC2cOFCHcsrm3fddRcAz+qtcKl4+/LLL73GVGUK\n4Hll88KFCwCA9957T4/NnDlTx4888oiOfVVUkDe5dqY8LkRFRXn9rKwoVE2BAODs2bNByq74eGWT\niIiIiIiILMeTTSIiIiIiIrJcWJTRyjJZtXZm/fr19diuXbt0LJsFUdH88MMPxX7ORx99pOMDBw7o\nuFq1ajpWJSxWOHjwIADg5Zdftuw17ZCUlKTj6tWr25hJ6Jk6daqOX3/9da/HZXOOgsphCyuTLezx\nadOmXfZxKjpZFi1jhaWz3tRtJJdSa5mmpqaaTCfsyBI3+V53+PBhAGwKZBd53JbvDb179wbguW7w\n/PnzdZyVlWUgu5Ljiy++0LH8LKWa1QwbNkyPySZwN99882VfVzYmo4vkbTZXXXWV1+Oq9B7wLP1O\nT08PbmJ+4pVNIiIiIiIishxPNomIiIiIiMhyYVFG27BhQx37Ws9HdjWVJbV0kerQq0pKrNC3b98i\n/6zqdAb4LlNcsmSJjgtaG27t2rXFyK7kSk5O1rEqD8/IyNBj33zzjfGcQoVcc+qJJ57QcXR0tGXb\nOHLkiI63b98OALj//vv1mCwZp8C43W6fMRWsS5cuPsezs7MBeHYJJ+vJMlq5zy5btszrZ2XpXOXK\nlXWsflcUHLJT9nPPPQcAGDdunB4bO3asjgcOHAggfNb0Ve9pgGcHX7mWsqLWKL2UWiVA7vNPPfWU\nVSmGNPlv/sknn7zsz86dO1fHaWlpwUrJMryySURERERERJbjySYRERERERFZzrFltPXq1dOx7KCl\nyDI62YWSvN1xxx0APC/rlylT5rLPadq0qY4L6yr77rvv6njPnj1ej3/yySc63rFjx2Vfy4muvPJK\nHXfr1s3r8Y8//ljHqkSFvO3du1fHKSkpOr799tsBACNHjgx4G7JD3+TJkwN+PSpYuXLlvMbCpZyt\nOOSxWt5SIv3+++8AgPPnzxvJiTyp4/aAAQP02KhRo3ScmZmp43vvvddcYmFu9uzZAIDhw4frMfV5\nCABeeOEFAP515A9F8vj6yCOP6DgqKgoAcMMNN+ixqlWr6lh+rpszZw4AYMyYMUHKMvSo+du2bZse\nK+gzttrX5PyHAl7ZJCIiIiIiIsu5gt1YweVy2dK5QV5hePrpp70eb926tY4LaipjF7fb7b1oXBHY\nNdehyt95BszOtfyG6+uvv9axWputf//+eiw3N9dUWsUSCvt0165ddSyb+sj1rlQzqhkzZugxucaj\n/GbSrkYeoTDXVlBr5wL567y9+OKLeszEmpGhMNdynel//etfOr7vvvt0rK7glOSrZqEw14WRzWfi\n4+N1rI4h8vPYO++8o2O5X+/bty+YKYbM+6JJdevW1bG8Sjdv3jwAnleki8MJ+7SkGiYBQNu2bXX8\n/PPP61h9bjGtJM+1Widz8eLFeqygc7Nbb70VALBmzZpgp+U3X3PNK5tERERERERkOZ5sEhERERER\nkeUcVUablJSkY7U2JJB/863EMlpiuZA53KfNCZe5/vzzz3U8fvx4AOZLi0JtrmvWrKnjl156Sceb\nN28GULKbWoXaXPsiP6Oo5jJA/vrIU6dO1WPHjh3T8blz5wxkdxHfFy9PNpy88cYbAQBt2rTRY/JW\nisI4YZ8OFSV5rr///nsAnqX1klzndfTo0cFOJ2AsoyUiIiIiIiIjeLJJRERERERElnPUOps33XST\njn2VzgLArl27AACnTp0ykhMREVlPdgmmovn55591PHjwYBszCU/r1q3TcYcOHWzMhPzVp08fHavy\nx0aNGumx4pTREgFAlSpVAHh2tpdde99++23jOVmNVzaJiIiIiIjIcjzZJCIiIiIiIss5qoy2IKrU\nAchfEPXXX3+1Kx0iIiIiCjEnTpzQcf369W3MhJxCdVNXfwPAiy++qOMDBw4Yz8lqvLJJRERERERE\nlnPUOptOUZLXA3ISridmDvdpczjX5nCuzeFcm8H3RXO4T5vDuTaH62wSERERERGRETzZJCIiIiIi\nIssFpUFQZGTkwd9//72aP88tV67coTNnzlS3Oien8neuOc/Fx7k2h3NtBo/V5nCuzeHxwxzOtTmc\nazN4rLZeUO7ZdLlcbn9f1+VyBXTPQLjxd645z8XHuTaHc20Gj9XmcK7N4fHDHM61OZxrM3isth7L\naImIiIiIiMhyPNkkIiIiIiIiy/Fkk4iIiIiIiCxn7GQz65csJE5PRIvpLZA4PREVX62ICf+ZYGrz\nYSXnRA46zOqAplOaIn5qPOc5iIYsHoJqb1RDs6nN7E7F8VbuXInGkxojZmIMXlv3mt3pOF6eOw8t\nprdAr3m97E7FsXj8MIv7tBmpG1IRPzWenz8MuPbta9F8WnMkTk9E65mt7U7H0bhf+8/YyWbM1THI\nGJ6B74Z/h833b0b5MuWR3DjZ1ObDSulSpTG+y3hkPpCJ9UPWY/L/m4wdR3fYnZYjDUochFV3r7I7\nDcfLc+dhxPIRWHX3KmQ+kIl5W+dxnw6y1A2paBLdxO40HI3HD7O4Twdf5uFMvJPxDjYN24Qtw7dg\nadZS7D622+60HKuUqxTS7k1DxvAMbBy20e50HIv7dWBsKaNdvXs1GlZpiDoV69ixecerHlUdCdUT\nAABRV0Qh9ppY7D+x3+asnCmpbhIql6tsdxqOt3H/Rlx39XWoV6keykSUQUpcChbvWGx3Wo6VcyIH\ny3cux9AWQ+1OxdF4/DCH+7QZ249uR5tabVC2dFlElIpA+3rtsWj7IrvTciw33Mhz59mdhuNxvw6M\nLSebC7YSi1lCAAADOElEQVQuQL+4fnZsOuzs+W0Pthzcgja129idCpHf9p/YjzoV8r+cql2hNvaf\n5BcowTJq1SiM6zQOLrCDOzkD92kz4qrGYW32Whw7cwy553Ox/P8vx77j++xOy7FccKHTnE5oNbMV\nZm6eaXc6jsX9OjClTW/w/B/nsSRrCV7t+KrpTYedU+dOoc/CPkjtmoqoK6LsToeIQsCyrGWoVr4a\nEqonIG1PGtywfi1mIpO4T5vT+JrGGP2n0eg0pxOirohCYvVERJSKsDstx0ofnI4aV9XAkdNH0GlO\nJ8RGxyKpbpLdaTkO9+vAGL+yuWLnCrSs0RLR5aNNbzqsXMi7gD4L+2Bgs4Ho3bi33ekQBaRWhVrI\nPp6t/zvnRA5qXVXLxoycK31fOpb8uAQNUhug3yf9sOanNbjn03vsTovIb9ynzRqUOAib7t+EtPvS\nUKlcJcRcHWN3So5V46oaAIDo8tFIbpyMjft532awcL/2n/GTzXlb57GE1oDBiwejSXQTjGw70u5U\nHM/9v/9R8LSq2Qo7f92Jvb/txbk/zmH+1vnodT07SgbD2FvHIntUNnaP3I35d85Hh/odMDt5tt1p\nORaPH8HHfdqsI6ePAACyj2fj0x2fon98f5szcqbc87k4de4UAOD0udP4YvcXiKsaZ3NWzsX92n9G\ny2hzz+di9e7VmNFjhsnNhp307HTM/e9cxFeNR+L0RLjgwthbx6Jro652p+Y4/T/pj7Q9afjlzC+o\n+1ZdPH/z8xiUOMjutBwnolQEJnWbhM4fdEaeOw9DEocgNjrW7rSIAsLjBznRnQvvxK9nfkWZiDKY\n0n0KKpStYHdKjnTo1CEkL0iGy+XChbwLGBA/AJ0bdrY7Lcfifu0/l9tt/TeqLpfL7e/rulwuuN1u\n3sFfRP7ONee5+DjX5nCuzeCx2hzOtTk8fpjDuTaHc20Gj9XWs6UbLRERERERETkbTzaJiIiIiIjI\ncjzZJCIiIiIiIsvxZJOIiIiIiIgsF5RutOXKlTvkcrmq+ftcq/NxMn/nmvNcfJxrczjXZvBYbQ7n\n2hweP8zhXJvDuTaDx2rrBaUbLREREREREYU3ltESERERERGR5f4PQ+8P2ZkutsAAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x29c4f6d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def make_image(X):\n",
    "    im = np.swapaxes(X.T, 0, 1)\n",
    "    im = im - im.min()\n",
    "    im = im * 1.0 / im.max()\n",
    "    return im\n",
    "\n",
    "plt.figure(figsize=(16, 5))\n",
    "for i in range(0, 10):\n",
    "    plt.subplot(1, 10, i+1)\n",
    "    plt.imshow(make_image(test_data[i][0]), interpolation='nearest', cmap=plt.get_cmap('gray'))\n",
    "    true = test_label[0][i]\n",
    "    pred = FPGA_predicted[i]\n",
    "    color = 'green' if true == pred else 'red'\n",
    "    plt.text(0, 0, true, color='black', bbox=dict(facecolor='white', alpha=1))\n",
    "    plt.text(0, 32, pred, color=color, bbox=dict(facecolor='white', alpha=1))\n",
    "\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ARM CPU Deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 18.4 s, sys: 1.21 s, total: 19.6 s\n",
      "Wall time: 1min\n"
     ]
    }
   ],
   "source": [
    "batch_size = 600\n",
    "%time prob = np.array(lasagne.layers.get_output(net['prob'], floatX(test_data[0:batch_size]), deterministic=True).eval())\n",
    "predicted = np.argmax(prob, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check our accuracy\n",
    "We expect around 90%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.985\n"
     ]
    }
   ],
   "source": [
    "accuracy = np.mean(predicted == test_label[0][0:batch_size])\n",
    "# print(predicted)\n",
    "# print(test_label[0][0:batch_size])\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
